An Inversion-Free Estimating Equation Approach for Gaussian Process Models

Gaussian processes are a common analysis tool in statistics and uncertainty quantification. The covariance function of the process is generally unknown and often assumed to fall into some parameteric class. One of the scalability bottlenecks for the largescale usage of these processes is the computation of the maximum likelihood estimates of the parameters of the covariance matrix. In a classical approach this requires a Cholesky factorization of the dense covariance matrix for each optimization iteration. Recent approaches with stochastic approximations of the score equations [1, 30, 29] require solving linear systems only with the covariance matrix, which is a significant improvement but continues to be a nontrivial expense. In this work, we present an estimating equation approach for the maximum likelihood estimation of parameters. The distinguishing feature of this approach is that no linear system needs to be solved with the covariance matrix. As a result, this approach requires only a small fraction of the computational e↵ort of maximum likelihood calculations; for certain commonly used covariance models and data configurations, this approach results in fast and scalable calculations. We prove that when the covariance matrix has a bounded condition number, our approach has the same convergence rate as does maximum likelihood in that the Godambe information matrix of the resulting estimator is at least as large as a fixed fraction of the Fisher information matrix. Moreover, our approach presents additional advantages compared with the previous ones [1, 30, 29], namely, the preservation of an optimization structure and the guarantee of finding global optima for covariance models that are linear in the parameters. We demonstrate the e↵ectiveness of the proposed approach on two synthetic examples, one of which involes up to 1 million data points. ⇤Preprint ANL/MCS-P5078-0214, Argonne National Laboratory. †Mathematics and Computer Science Division, Argonne National Laboratory, Argonne, IL 60439. Emails: (anitescu,jiechen)@mcs.anl.gov ‡Department of Statistics, University of Chicago, Chicago, IL 60637. Email: [email protected].